Web Survey Bibliography
In the literature on questionnaire design and survey methodology, pre-testing is mentioned as a way to evaluate questionnaires (i.e. investigate whether they work as intended) and control for measurement errors (i.e. assess data quality). As the American Statistical Association puts it (ASA, 1999, p. 11): “The questionnaire designer must understand the need to pretest, pretest, and then pretest some more.” Clark and Schober (1992, p. 29) indicate why this need to pre-test: “Surveyors cannot possibly write perfect questions, self-evident to each respondent, that never need clarification. And because they cannot, the answers will often be surprising.”
In this Ph.D. thesis I have tried to systematically describe my experiences with pre-test research at the Questionnaire Laboratory at Statistics Netherlands, a cognitive laboratory which started its work in 1992. This text is not aimed at a theoretical discussion of cognitive laboratory methods, but focuses on the application of these methods: setting-up and carrying-out pre-test research, analysing the data and presentation of the results.
The thesis starts with an introduction of cognitive laboratory research, including the history of the CASM (Cognitive Aspects of Survey Methodology) movement and the history of the Questionnaire Laboratory at Statistics Netherlands. The next two chapters address aspects of computer-assisted interviewing. Since at Statistics Netherlands most social-survey questionnaires are computerised, this sets the conditions for pre-test research at the Questionnaire Laboratory. Chapter 2 discusses computer-assisted interviewing in general; chapter 3 addresses the effects of computer-assisted interviewing on data quality. These three chapters are introductory to the chapters that follow, the actual core of the thesis, in which the application of cognitive laboratory methods, including several case studies, are discussed.
The methods used at the Questionnaire Laboratory at Statistics Netherlands are discussed in chapters 4 and 5. Chapter 4 presents an overview of pre-test methods. Expert appraisal, focus groups, in-depth interviewing (including follow-up probing, meaning-oriented probing, paraphrasing, targeted test questions, and vignettes), and behavioural coding are discussed from a practical point of view, i.e. how they are applied in the Questionnaire Laboratory. Computer-Assisted Qualitative Interviewing (CAQI) is discussed in chapter 5. The CAQI method has been developed at the Questionnaire Laboratory to pre-test computerised questionnaires. With CAQI a pre-test protocol is integrated in a computerised questionnaire to be tested.
The next four chapters present case studies of cognitive research in which the methods addressed in the chapter 4 and 5 have been applied. These chapters discuss the design and the results (i.e. identified problems in the questionnaire and recommendations for improvement) of these studies.
Chapter 10 concludes this thesis with a summary. A number of identified problems in the investigated questionnaires are: vague or unclear wording, complex syntax, long question, double-barrelled question, conflict with previous question(s), question asks for specific information that is not available by heart, difficult to come to an answer because of complex calculation, overlapping or missing response items. In the conclusion, the identified problems are related to design errors in the questionnaire.
igitur Homepage (abstract) / (full text)
Web survey bibliography (4086)
- Around the World Wide Web in 80 Ways: How Motives for Going Online are Linked to Internet Activities...; 2003; Johnson, T. J., Kaye, B. K.
- Comparing On-Line and On-Paper Responses to an Employee Satisfaction Survey; 2003; Stone, G. E.
- Express: A Web-based technology to support human and computational experimentation; 2003; Yule, P., Couper, M. P.
- An Evaluation of the Semiautomatic Login Procedure to Control Web Survey Access; 2003; Heerwegh, D., Loosveldt, G.
- Web site design benchmarking within industry groups; 2003; Kim, S.-E., Shaw, T., Schneider, H.
- Identifying key factors affecting consumer purchase behavior in an online shopping context; 2003; Park, C.-H., Kim, Y.-G.
- Using Web Surveys To Reach Community College Students: An Analysis of Response Rates and Response Bias...; 2003; Sax, L. J., Gilmartin, S. K., Hagedorn, L. S., Lee, J. J.
- Internet Recruiting: The Effects of Web Page Design Features; 2003; Braddy, P. W., Foster Thompson, L., Wuensch, K. L., Grossnickle, W. F.
- An Experimental Comparison of Computer-Mediated and Face-to-Face Focus Groups; 2003; Underhill, C., Olmsted, M. G.
- Exploiting Hyperlinks to Study Academic Web Use; 2003; Wilkinson, D., Thelwall, M., Li, X.
- Who Participates and Why?; 2003; Weber, L. M., Loumakis, A., Bergman, J.
- Utility of Web-based assessment of patient satisfaction with endoscopy; 2003; Harewood, G. C., Wiersema, M. J., de Groen, P. C.
- We seek them here, we seek them there. How technical innovation in mixed mode survey software is responding...; 2003; Macer, T.
- Costs and benefits of improving response rates for a hard-to-reach population; 2003; Teitler, J. O., Reichman, N. E.,
- Humanizing self-administered surveys: experiments on social presence in web and IVR surveys; 2003; Tourangeau, R., Couper, M. P., Steiger, D. M., de Rouvray, C.
- Web Surveys as Part of a Mixed-Mode Strategy for Populations That Cannot Be Contacted by E-Mail; 2003; Schonlau, M., Asch, B. J., Du, C.
- Web Survey's Hidden Hazards; 2003; Morrel Samuels, P.
- Using pencil and paper, Internet and touch-tone phones for self-administered surveys: does methodology...; 2003; Knapp, H., Kirk, S. A.
- IMRO Online Survey Satisfaction Research: A Pilot Study of Salience-based Respondent Experience Modeling...; 2003; MacElroy, B., Gray, Mi.
- Validations in Web-based Surveys; 2003; Crawford, S. D., Peytchev, A.
- Use and non-use of clarification features in web surveys; 2003; Tourangeau, R., P., Couper, M. P., Conrad, F. G., Baker, R. P.
- Online versus postal data collection methods: an examination of issues and a comparison of results; 2003; Adam, S., McDonald, H.
- Assessing Response Rates And Nonresponse Bias In Web And Paper Surveys; 2003; Sax, L. J., Gilmartin, S. K., Bryant, A. N.
- Asking Comparative Questions: A Comparison Of Three Wording Strategies; 2003; Liu, K.
- It's Only Incidental: Effects of Response Format in Determining Behavioral or Event Occurrence; 2003; Lafond, C. R., Smith, M. R., Behnke, C. S., Thomas, R. K.
- Achieving high response rates on web-based surveys of post-secondary students; 2003; Nichols, L. B., Ghadialy, R.
- Design implementation of a Multimode Web Survey; 2003; Wine, S. J., Cominole, M. B., Carwile, D. S., Perry, K.
- Partnering with a Newspaper to Assess Community Opinion Online; 2003; Downs, E. P., Lindley, A. M.
- Implementing a Web Survey Administration System at the GAO; 2003; Feldesman, A. G.
- Using Internet-Based Surveys With Physicians, What Works and What Doesn't Work; 2003; Schneiderman, M., Thran, S., Adams, C., Lerner, B.
- Exploring Online Survey Metodologies: Who are the Respondents and How to Get them to respond; 2003; Wolter-Warmerdam, K., Gardinali, A. P., Wong, R.
- Using RGI (Respondent Generated Interval) to gather factual information in a web survey; 2003; Lusinchi, D.
- Can What We Don’t Know (about “Don’t Know”) Hurt Us?: Effects of Item Non-response...; 2003; Krosnick, J. A., Behnke, C. S., Lafond, C. R., Thomas, R. K.
- Survey Administration Effects? A Comparison of Web-Based and Traditional Written Self-Administered Surveys...; 2003; Bandilla, W., Bosnjak, M., Altdorfer, P.
- What They See Is What We Get: Response Options for Web Surveys; 2003; Couper, M. P., Tourangeau, R., Conrad, F. G., Crawford, S. D.
- Web Survey Design: Comparing Static and Interactive Survey Instruments; 2003; Nyiri, Z., Clark, L. R.
- Comparing Web-Based Survey Methods with other Approaches: An examination of health knowledge, opinion...; 2003; Greiling, K. A., McCarrier, P. K., Stringer, M. C.
- The role of issue involvement in UK public attitudes to the single European currency; 2003; Roberts, E. C.
- Partipation in Online Surveys: Results from a Series of Experiments; 2003; Kiniorski, A. K., R.Smith, M. R.
- Does Order Really Make a Difference? The Impact of Respondent and Question Characteristics on Response...; 2003; O'Neill, G.
- The Advent of internet surveys for political research: a comparison of telephone and internet surveys...; 2003; Berrens, R. P., Bohara, A. K., Jenkins-Smith, H. C., Silva, C. L., Weimer, D. L.
- The Global Digital Divide: A Sociological Assessment of Trends and Causes; 2003; Drori, G. S., Jang, Y. S.
- On-line qualitative market research: Interviewing the world at a fingertip; 2002; Scholl, N., Mulders, S., Drent, R.
- A lesson model: Using the Web for research; 2002; Grant, S.
- Willingness to Pay for Organic Foods: A Comparison between Survey Data and Panel Data from Denmark ; 2002; Millock, K.; Hansen, L.G.; Wier, M.; Andersen, L.M
- Item Nonresponse: Distinguishing between don't Know and Refuse; 2002; Pamela J. Shoemaker, Martin Eichholz and Elizabeth A. Skewes
- The Differential Effects of Face-to-Face and Computer Interview Modes; 2002; Cooley, P. C.; Des Jarlais, D.C.; Gribble, J.; Newman, J.C.; F.; Paone, D.; Turner, C. F.
- Group decision making in a multiple criteria environment: A case using the AHP in software selection; 2002; Lai, V.S., Wong, B.K., Cheung, W.
- Cognitive Laboratory Experiences : On Pre-testing Computerised Questionnaires; 2002; Snijkers, G.
- Methodological Issues In Web-based Research; 2002; Duffy, M. E.